DOP: Deep Optimistic Planning with Approximate Value Function Evaluation

Research on reinforcement learning has demonstrated promising results in manifold applications and domains. Still, efficiently learning effective robot behaviors is very difficult, due to unstructured scenarios, high uncertainties, and large state dimensionality (e.g. multi-agent systems or hyper-redundant robots). To alleviate this problem, we present DOP, a deep model-based reinforcement learning algorithm, which exploits action values to both (1) guide the exploration of the state space and (2) plan effective policies. Specifically, we exploit deep neural networks to learn Q-functions that are used to attack the curse of dimensionality during a Monte-Carlo tree search. Our algorithm, in fact, constructs upper confidence bounds on the learned value function to select actions optimistically. We implement and evaluate DOP on different scenarios: (1) a cooperative navigation problem, (2) a fetching task for a 7-DOF KUKA robot, and (3) a human-robot handover with a humanoid robot (both in simulation and real). The obtained results show the effectiveness of DOP in the chosen applications, where action values drive the exploration and reduce the computational demand of the planning process while achieving good performance

Q-CP: Learning Action Values for Cooperative Planning

Research on multi-robot systems has demonstrated promising results in manifold applications and domains. Still, efficiently learning an effective robot behaviors is very difficult, due to unstructured scenarios, high uncertainties, and large state dimensionality (e.g. hyper-redundant and groups of robot). To alleviate this problem, we present Q-CP a cooperative model-based reinforcement learning algorithm, which exploits action values to both (1) guide the exploration of the state space and (2) generate effective policies. Specifically, we exploit Q-learning to attack the curse-of-dimensionality in the iterations of a Monte-Carlo Tree Search. We implement and evaluate Q-CP on different stochastic cooperative (general-sum) games: (1) a simple cooperative navigation problem among 3 robots, (2) a cooperation scenario between a pair of KUKA YouBots performing hand-overs, and (3) a coordination task between two mobile robots entering a door. The obtained results show the effectiveness of Q-CP in the chosen applications, where action values drive the exploration and reduce the computational demand of the planning process while achieving good performance

Spatial representation for planning and executing robot behaviors in complex environments

Robots are already improving our well-being and productivity in different applications such as industry, health-care and indoor service applications. However, we are still far from developing (and releasing) a fully functional robotic agent that can autonomously survive in tasks that require human-level cognitive capabilities. Robotic systems on the market, in fact, are designed to address specific applications, and can only run pre-defined behaviors to robustly repeat few tasks (e.g., assembling objects parts, vacuum cleaning). They internal representation of the world is usually constrained to the task they are performing, and does not allows for generalization to other scenarios. Unfortunately, such a paradigm only apply to a very limited set of domains, where the environment can be assumed to be static, and its dynamics can be handled before deployment. Additionally, robots configured in this way will eventually fail if their "handcrafted'' representation of the environment does not match the external world. Hence, to enable more sophisticated cognitive skills, we investigate how to design robots to properly represent the environment and behave accordingly. To this end, we formalize a representation of the environment that enhances the robot spatial knowledge to explicitly include a representation of its own actions. Spatial knowledge constitutes the core of the robot understanding of the environment, however it is not sufficient to represent what the robot is capable to do in it. To overcome such a limitation, we formalize SK4R, a spatial knowledge representation for robots which enhances spatial knowledge with a novel and "functional" point of view that explicitly models robot actions. To this end, we exploit the concept of affordances, introduced to express opportunities (actions) that objects offer to an agent. To encode affordances within SK4R, we define the "affordance semantics" of actions that is used to annotate an environment, and to represent to which extent robot actions support goal-oriented behaviors. We demonstrate the benefits of a functional representation of the environment in multiple robotic scenarios that traverse and contribute different research topics relating to: robot knowledge representations, social robotics, multi-robot systems and robot learning and planning. We show how a domain-specific representation, that explicitly encodes affordance semantics, provides the robot with a more concrete understanding of the environment and of the effects that its actions have on it. The goal of our work is to design an agent that will no longer execute an action, because of mere pre-defined routine, rather, it will execute an actions because it "knows'' that the resulting state leads one step closer to success in its task

S-AVE Semantic Active Vision Exploration and Mapping of Indoor Environments for Mobile Robots

Semantic mapping is fundamental to enable cognition and high-level planning in robotics. It is a difficult task due to generalization to different scenarios and sensory data types. Hence, most techniques do not obtain a rich and accurate semantic map of the environment and of the objects therein. To tackle this issue we present a novel approach that exploits active vision and drives environment exploration aiming at improving the quality of the semantic map

A Tool for Encoding Controlled Natural Language Specifications as ASP Rules.

Answer Set Programming (ASP) is a popular declarative programming language for solving hard combinatorial problems. Albeit ASP has been widely adopted in both academic and industrial contexts, it might be difficult for people who are not familiar with logic programming conventions to use it. In this paper, we propose a translation of English sentences expressed in a controlled natural language (CNL) form into ASP. In particular, we first provide a definition of the type of sentences allowed by our CNL and their translation as ASP rules, and then exemplify the usage of CNL for the specification of well-known combinatorial problems

DOP: Deep Optimistic Planning with Approximate Value Function Evaluation

Research on reinforcement learning has demonstrated promising results in manifold applications and domains. Still, efficiently learning effective robot behaviors is very difficult, due to unstructured scenarios, high uncertainties, and large state dimensionality (e.g. multi-agent systems or hyper-redundant robots). To alleviate this problem, we present DOP, a deep model-based reinforcement learning algorithm, which exploits action values to both (1) guide the exploration of the state space and (2) plan effective policies. Specifically, we exploit deep neural networks to learn Q-functions that are used to attack the curse of dimensionality during a Monte-Carlo tree search. Our algorithm, in fact, constructs upper confidence bounds on the learned value function to select actions optimistically. We implement and evaluate DOP on different scenarios: (1) a cooperative navigation problem, (2) a fetching task for a 7-DOF KUKA robot, and (3) a human-robot handover with a humanoid robot (both in simulation and real). The obtained results show the effectiveness of DOP in the chosen applications, where action values drive the exploration and reduce the computational demand of the planning process while achieving good performance.Comment: to appear as an extended abstract paper in the Proc. of the 17th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2018), Stockholm, Sweden, July 10-15, 2018, IFAAMAS. arXiv admin note: text overlap with arXiv:1803.0029

Stiffened panels damage tolerance determination using an optimization procedure based on a linear delamination growth approach

The damage tolerance of delaminated composite panels under compressive load is usually numerically evaluated by means of computationally expensive non-linear approaches. In this study, an alternative numerical linear approach, able to mimic the delamination propagation initiation, is proposed. With the aim to exploit its benefits, in terms of computational costs reduction, the proposed linear methodology has been used in this study in conjunction with an optimization analysis to assess the damage tolerance of stiffened composite panels with an impact induced delamination under compression. Indeed, the optimization was aimed to find the minimum delamination growth initiation load for a delaminated stiffened panel with variable delamination size and position, providing indications on the damage tolerance capability of the stiffened panel with an arbitrary positioned and sized delamination induced (as an example) by a low energy impact

Simple and Rapid Non-Enzymatic Procedure Allows the Isolation of Structurally Preserved Connective Tissue Micro-Fragments Enriched with SVF

The stromal vascular fraction (SVF) consists of a heterogeneous population of stem and stromal cells, generally obtained from adipose tissue by enzymatic digestion. For human cell-based therapies, mechanical process methods to obtain SVF represent an advantageous approach because they have fewer regulatory restrictions for their clinical use. The aim of this study was to characterize a novel commercial system for obtaining SVF from adipose tissue by a mechanical approach without substantial manipulations. Lipoaspirate samples collected from 27 informed patients were processed by a simple and fast mechanical system (by means of Hy-Tissue SVF). The Hy-Tissue SVF product contained a free cell fraction and micro-fragments of stromal connective tissue. The enzymatic digestion of the micro-fragments increased the yield of free cells (3.2 times) and CFU-F (2.4 times). Additionally, 10% of free cells from SVF were positive for CD34+, suggesting the presence of endothelial cells, pericytes, and potential adipose-derived stem cells (ADSC). Moreover, the SVF cells were able to proliferate and differentiate in vitro toward adipocytes, osteocytes, and chondrocytes. The immunophenotypic analysis of expanded cells showed positivity for typical mesenchymal stem cell markers. The Hy-Tissue SVF system allows the isolation of stromal vascular fraction, making this product of potential interest in regenerative medicine

Face Authentication using Speed Fractal Technique

In this paper, a new fractal based recognition method, Face Authentication using Speed Fractal Technique (FAST), is presented. The main contribution is the good compromise between memory requirements, execution time and recognition ratio. FAST is based on Iterated Function Systems (IFS) theory, largely studied in still image compression and indexing, but not yet widely used for face recognition. Indeed, Fractals are well known to be invariant to a large set of global transformations. FAST is robust with respect to meaningful variations in facial expression and to the small changes of illumination and pose. Another advantage of the FAST strategy consists in the speed up that it introduces. The typical slowness of fractal image compression is avoided by exploiting only the indexing phase, which requires time O(D log (D)), where D is the size of the domain pool. Lastly, the FAST algorithm compares well to a large set of other recognition methods, as underlined in the experimental results

BIRD: Watershed Based IRis Detection for mobile devices

Communications with a central iris database system using common wireless technologies, such as tablets and smartphones, and iris acquisition out of the field are important functionalities and capabilities of a mobile iris identification device. However, when images are acquired by means of mobile devices under uncontrolled acquisition conditions, noisy images are produced and the effectiveness of the iris recognition system is significantly conditioned. This paper proposes a technique based on watershed transform for iris detection in noisy images captured by mobile devices. The method exploits the information related to limbus to segment the periocular region and merges its score with the iris' one to achieve greater accuracy in the recognition phase